Harvesting Collective Agreement in Community Oriented Surveys: The Medical Case

The chapter discusses the role of simple and lightweight Web-based systems in promoting a different approach to the externalization of practice-related knowledge within communities of professionals. This approach exploits common online questionnaire systems to collect the preferences of large numbers of domain experts to interesting paradigmatic work cases and proposes a statistically sound evaluation of these responses to evaluate the agreement reached within the community. We tested this approach in a case study that involved a large international medical association, that we chose as an example of a large and highly distributed community of expert professionals; in this study we challenged more than 1,000 surgeons about some border-line clinical cases where tacit notions based on lifelong practice and situated experiences coexist (and sometimes clash) with scientific evidences drawn from the specialistic literature. We make the point that a sound evaluation of the collective agreement is a necessary precondition to use such lean Web-based tools in bottom-up knowledge elicitation initiatives. To this aim, existing measures of agreement and survey-related heuristics can be exploited to get a more precise picture of the “opinion of the many” in collective settings like communities of practice. 1 Background and motivations Surveys and polls are important means to collect information since the beginning of the nineteenth century [20] and the rationalization of both the instruments and methods to process, analyze and interpret the collected data can be said coeval to the development of the machines that were to evolve in the modern programmable computers [21]. Surveys have been used extensively in different disciplines, like sociology, psychology, economics, education and also in medicine. In this domain, Università degli Studi di Milano-Bicocca, Viale Sarca 336, I 20126 Milano (Italy), e-mail: [email protected]